Method and device for virtualization of multiple data sets on same associative memory

ABSTRACT

A system ( 200 ) can provide data aggregation with a single primary table ( 206 ) formed in a content addressable memory (CAM) section ( 202 ). Within a primary table ( 206 ) CAM entries can be part of a primary table, one or more aggregate tables, or both. In one arrangement, valid bits in each CAM entry can indicate which particular schemes a CAM entry belongs to (primary table, or any of the aggregate tables). Associated data for each table can be stored in a RAM section ( 204 ) and can be accessed according to an offset address generated according to a scheme value (i).

TECHNICAL FIELD

The present invention relates generally to associative memories, such ascontent addressable memory (CAM) devices, and more particularly tomethods and devices for projecting multiple data sets onto the same setof associative memory entries.

BACKGROUND OF THE INVENTION

As is well known, data networks can enable the flow of data packetsbetween network source(s) and destination(s). Such applications caninclude packet forwarding applications, such as the generation of a“next hop” address as a data packet propagates through a network.Further, to provide additional services, increase performance, andmanage growth, it can be desirable to acquire data regarding networkuse. As but one example, it can be desirable to measure network flowdata.

Network flow measurement typically includes aggregating particular datavalues related to each network flow. This collected data can then beused for various applications, including but not limited to, billing,traffic management, and other types of network analysis techniques. Anumber of network flow measurement techniques are known. Threeparticular naming conventions are “NetFlow” provided by Cisco Systems,of San Jose, Calif., “J-Flow” provided by Juniper Networks, ofSunnyvale, Calif., and “sFlow” provided by Foundry Networks, Inc., ofSan Jose, Calif.

To better understand features and advantages of the disclosedembodiments, examples of conventional network data aggregation will nowbe described. FIG. 23 shows an example of a table containing data fornetwork flows, as well as associated data for such flows. FIG. 23 is adiagram depicting a “Primary Table” for storing flow information, andrepresents one very particular approach in which five fields areutilized to define a network flow: a Source IP Address, a Destination IPAddress, a Protocol Type, a Source Port, and a Destination Port. In suchan arrangement, these fields can be collectively considered a “flowdescriptor”. As is well understood, such fields can exist in a networkpacket header transmitted according to a given protocol (e.g., TCP/IP).The associated data of FIG. 23 shows two fields: Number of Packets andBytes.

Of course, FIG. 23 represents but one example of representing oneparticular flow type. Packets transmitted according to differentprotocols would have different header information. For example, NetFlownoted above utilizes seven fields to define a flow.

Referring still to FIG. 23, in a conventional data aggregation approach,an incoming packet header can be parsed for the network flow descriptorfields (in this example, five fields). If these fields of the packet areunique (i.e., not yet included as an entry in the Primary Table), a newentry is created for the flow, with appropriate Associated Data. Forexample, a “Number of Packets” can be set to 1, and the “Bytes” can bethe number of bytes in the received packet. If the flow descriptorfields of a received packet are not unique, the data associated with theflow can be updated according to the received packet (i.e., number ofpackets and bytes is incremented according to the new packet).

The amount of data produced by flow identification can be very large,due to the granularity with which flows are typically defined (number offields for flow descriptor). That is, the larger the number of fieldsused to define flows, the more granularity in the flow data. Highgranularity data can create a data explosion in network statistics thatcan be difficult to manage and analyze.

Granularity of accumulated flow data is typically defined by networkequipment provided by a vendor. In many cases, the granularity providedby a vendor can be higher than necessary for a given application. Thatis, a given application may need to collect statistics on a smallernumber of fields (e.g., two fields) than the granularity provided by thenetwork equipment. In such applications, collected network flow data canbe aggregated based on the smaller number of fields.

Network flow data aggregation can address the large amounts of datapresented by high granularity flow data. By aggregating data based on asmaller number of fields, concise flow data can be gathered that iseasier to interpret and transfer over a network. For example, CiscoNetFlow (Version 8) supports eleven aggregation schemes for addressingdata explosion by making the amount of flow data more tractable.

The number of flows in a network at a given instant can be a very largenumber that varies as newer flows are added and older flows are deleted.Thus, entries of a Primary Table, like that of FIG. 23, can becontinuously aged (deleted, expired) at an average flow rate in order tofree up space for new flows. Aging of Primary Table entries can be basedon various criteria. A few rules for aging according to NetFlow include(1) expiring idle flows (flows that have been idle for a specifiedamount of time are expired and removed from the primary table); (2)expiring long lived flows (e.g., as a default, flows in existence for 30minutes can be expired); (3) expiring flows as the Primary Table fillsup (as the Primary Table fills up, a number of heuristics can be appliedto aggressively age groups of flows simultaneously); (4) expiring flowsbased on flag detection (flows can be expired based on predeterminedflags/indications within a packet (e.g., for TCP end of byte stream flag(FIN) or reset flag (RST)).

In conventional arrangements, there can be two possible outcomes forPrimary Table entry that is aged. These operations are shown in FIG. 24.FIG. 24 is a diagram that shows Primary Table 2400 of FIG. 23, whichincludes flow-defining entries 2402-1 to 2402-4. In FIG. 24, it isassumed that the first entry 2402-1 is aged (expired from the PrimaryTable).

A first outcome can occur if the network equipment does not support dataaggregation. In this case, the associated data for the aged flow can besent to a flow collector device 2404.

However, if data aggregation is supported, associated data for theexpired entry can be added one or more Secondary Tables (or aggregationtables). The example of FIG. 24 shows two aggregation schemes, resultingin two Secondary Tables 2406-0 and 2406-1. In particular, SecondaryTable #1 2406-0 aggregates flow data based on Source IP Address andDestination IP Address, while Secondary Table #2 2406-1 aggregates flowdata based on Source IP Address and Source Port.

In such an arrangement, as a Primary Table entry is expired, the SourceIP Address and Destination IP Address are checked against correspondingfields in Secondary Table #1 2406-0. If the Source IPAddress/Destination IP Address combination is unique, a new entry isadded to Secondary Table #1 2406-0. If the combination already exists,the associated data for the Secondary Table #1 2406-0 can be updated. Inthe same general fashion, a Source IP Address and Source Port Addresscan be checked against corresponding fields in Secondary Table #2 2406-1and either a new entry added or the table updated.

A number of conventional approaches to aggregating data are known.

One approach for aggregating data utilizes “hashing”. One example of ahashing approach is shown in FIG. 25, and designated by the generalreference character 2500. In a hashing arrangement, a Primary Table canbe stored in RAM 2502. When an entry is expired, the fields of the entrycorresponding to an aggregation scheme can be applied to a hash functionto arrive at an address at which to store aggregation data. Moreparticularly, network flow data can be stored in a random access memory(RAM), and hashing can be used to differentiate between primary tableand aggregate table entries. In the particular example shown, a hashingfunction can be executed by hashing logic 2504 formed in an applicationspecific integrated circuit (ASIC) 2506.

A drawback to conventional hashing approaches can be the amount ofmemory space needed to accommodate the different data tables. That is, asingle entry is needed for each value of the different tables.

Another drawback to conventional hashing approaches can be “collisions”.As is well known, hashing functions can map a larger address space(i.e., that represented by all possible key combinations) into a smalleraddress space (i.e., that of the RAM). However, hashing functions arerarely perfect and thus give rise to collisions in which two keys hashto the same location. In such a case, the colliding key must bere-checked with each colliding entry to complete a search. Due thevariations in key values, collisions can be an inevitable problem withhashing solutions, and is anticipated to become an even more criticalproblem as data aggregation is performed at faster line rates (bittransmission speeds).

The above collision problem also gives rise to non-deterministic searchtimes. That is, while a non-colliding search may take one memory access,a colliding search may make multiple memory accesses. It is believedsuch varying search times will also be more difficult to handle as linerates increase.

Another drawback to hashing solutions can be the data dependenceinherent in hashing functions. In particular, a hashing function willgive a different collision profile depending upon the input data set.That is, a hashing function operating on a randomly generated data setmay have a different collision profile than one operating on actualnetwork data. Because of this, for optimal performance, a hash functionpolynomial is optimally selected based on a known entry format andexpected distribution. However, such approaches remain imperfect.

Related to the above hashing function mapping problems is “funneling”.Funneling is a flaw in some hashing functions that can arise when inputvalues vary by only a small amount (i.e., differ by only a few bits). Insuch cases, variability based on input bit values can be lost. This canlead to entries that vary by only a few bits hashing to the same value,thus giving rise to a large number of collisions.

A second conventional approach is shown in FIG. 26 and designated by thegeneral reference character 2600. A conventional arrangement 2600 caninclude a CAM portion 2602 and a dynamic RAM (DRAM) portion 2604. In theconventional example of FIG. 26, key portions of both a Primary Table(containing network flow entries) and Secondary Tables (containingaggregate data entries) can be stored in one or more CAM sections 2602.Associated data for each CAM entry can be stored in DRAM portion 2604.In a search operation, a matching CAM entry will generate an index valuefor accessing the RAM entry containing the associated data.

A conventional approach like that of FIG. 26 can offer considerableadvantages in terms of speed and use over hashing approaches.Utilization of CAMs can result in search times that are deterministicand very fast, due to the high speed at which CAM devices can comparekey values.

However, a conventional approach like that of FIG. 26 is not withoutdrawbacks. While CAMs provide advantageously fast search speeds, thecost per bit for a CAM device can be considerably higher than that of aDRAM or static RAM (SRAM). As would be understood from FIG. 26, if thenumber of secondary tables is high, the cost would increasecorrespondingly.

To better understand various features of the disclosed embodiments, acomparison with respect to the memory requirements of the conventionalapproaches noted above will be discussed in more detail. Due to theabsence of predictability in the behavior of certain networks, and toensure optimal performance, each secondary table would generally have tobe at least equal in size to the primary table. This is because eachentry of a primary table could potentially aggregate to a unique entryin a secondary table. Accordingly, if a networking device has a primarytable of size “N” entries and has “m” aggregation schemes, the memoryspace needed would be N*(m+1). It is noted that in a hashing arrangementlike that of FIG. 25, while a RAM could include N*(m+1) entries forstoring data table values, in order to minimize collisions, a memoryspace allocated for the entries is typically at least 2× or 4× thenumber of anticipated data values. Thus, an actual implementation couldrequire an overall storage space size of 4*N*(m+1), with “m+1” beingrepresenting the primary table and “m” aggregation schemes.

In the case of a CAM/RAM approach like that of FIG. 26, a system wouldneed both N*(m+1) CAM entries, as well as N*(m+1) DRAM entries.

In light of the above, it would be desirable to arrive at some wayaggregating network flow data that does not require as much CAM memoryas conventional approaches like those described above. However, at thesame time, such a solution should not suffer from the non-deterministicbehavior that can arise from approaches utilizing hashing functions.

In the case of packet forwarding, previously forwarding functions couldbe based on a single criterion: the destination address of a datapacket. However, presently more sophisticated forwarding approaches areneeded and/or anticipated. That is, it is desirable to forward datapackets based on multiple criteria.

As but one very particular example, it is desirable to accommodatevarying levels of service for data packets based on one or moreidentifying features, e.g., different “quality of service” (QOS) or“type of service” (TOS). Conventionally, approaches to such require verylarge lookup data bases, increasing components size and hence systemcost. In particular, a conventional approach can include multipleentries (e.g. CAM storage locations) for each destination/servicecombination.

In light of the above, it would be desirable to arrive at some way ofreducing the number of system components needed in forwarding operationsbased on multiple criteria.

Still further, all of the above illustrates how there is a general needto arrive at some way of reducing the storage space needed forassociative memory (e.g., CAM) applications, particularly thoseembodying multiple data sets.

SUMMARY OF THE INVENTION

The present invention can include a method of providing multiple datasets for search on a single set of storage entries. The method caninclude storing at least one data value of a first data set in onephysical entry of a plurality of globally maskable content addressablememory (CAM) entries and storing at least a second data value of asecond data set, different form the first data set, in the one physicalentry. The method can also include distinguishing between the first andsecond data value by applying a predetermined global mask to the oneglobally maskable CAM entry.

In such an arrangement a single CAM entry can serve to store data valuesfor two or more data sets. That is, one set of CAM entries willvirtually include multiple data sets. This can reduce the number ofentries needed to represent such multiple sets with respect toconventional arrangements that provide a different set of CAM entriesfor each data set.

According to another aspect of the embodiments, the method can furtherinclude storing associated data for each data value of the first dataset in a first memory section starting a first offset location andstoring associated data for each data value of the second data set in asecond memory section having different addresses than the first memorysection, the second memory section starting a second offset location.

In such an arrangement one CAM entry can point to two differentassociated data sections.

According to another aspect of the embodiments, a first data set caninclude a network flow primary table and a second data set can compriseat least one aggregation scheme. A step of storing at least one datavalue of a first data set can include storing network flow informationin corresponding physical CAM entries to form network flow entries ofthe primary table. A step of storing a second data value can includeusing an existing primary network flow entry as an aggregation entrythat corresponds to the at least one aggregation scheme. The method canfurther include storing associated data for each flow entry in a firstassociated data area and storing associated data for each aggregationscheme in a different associated data area.

By utilizing existing network flow entries as aggregation entries,secondary (e.g., aggregation) table entries can be “superimposed” onto aprimary table. This can eliminate the need to provide CAM entries forall aggregation schemes, and reduce the CAM requirements of a system toa single primary table.

According to another aspect of the embodiments, storing network flowinformation can further include setting a primary table valid bit foreach network flow entry.

Such an arrangement can distinguish between primary and aggregationentries with valid bits.

According to another aspect of the embodiments, a primary table can haveprimary table entries that each include “k” data fields, where k is aninteger greater than 1. Each aggregation scheme can include a subset “p”of the k data fields, where p<k. Distinguishing between the primary andsecondary table values can include searching while applying a schemeglobal mask value that masks all but the p data fields of the k datafields to search the aggregation scheme, and searching without thescheme global mask value to search the primary table.

In this arrangement, an aggregation scheme can be projected onto networkflow entries.

According to another aspect of the embodiments, storing network flowinformation can include receiving new network flow information, andsearching for an existing aggregation scheme entry having a subset offields that match the new network flow information fields. Assuming anaggregation scheme entry is found, if writing the new network flowinformation would not alter a field value combination utilized by theexisting aggregation scheme entry, the method can include storing thenew network flow information in the matching CAM entry. However, ifwriting the new network flow information would alter the field valuecombination utilized by the existing aggregation scheme entry, themethod can include storing the new network flow information in an unusedCAM entry, if an unused CAM entry is available.

In this way, a compact primary table can be maintained as new entriesmay be written into existing aggregate entries, and will only require anunused entry if such an operation is not possible. According to anotheraspect of the embodiments, the method can include receiving a deletecommand to delete an existing primary table entry and aggregate the dataof the primary table entry. In response to the delete command, themethod can include, for each of a plurality of aggregation schemes,searching for an existing aggregation entry having a subset of fieldsthat match corresponding fields of the primary table entry. If anexisting aggregation entry cannot be found that has the subset of fieldsthat match corresponding fields of the primary table entry, convertingsaid network flow entry into an aggregation scheme entry. In this way, acompact primary table can be maintained, as a new aggregate entry can becreated for only those aggregation schemes not yet in existence by usinga network flow entry.

According to another aspect of the embodiments, flow entries can bedistinguished from aggregation entries according to valid bits in eachentry. In one very particular case, valid bits can include at least oneunique bit location corresponding to a primary table (for flow entries),and at least one bit location unique to each aggregation scheme.

Such an arrangement can provide a dependable and compact way of“superimposing” primary and aggregate entries in the same set of CAMentries.

According to another aspect of the embodiments, the method can includeaccessing associated data for each aggregation entry by generating anoffset address according to a scheme value, and adding the offsetaddress to an index value that corresponds to a matching aggregationentry.

In this way, associated data for both the primary and aggregate tablescan accessed with a single CAM entry table.

According to another aspect of the embodiments, a first data set caninclude a first type service designation for a data packet and seconddata set can include a second type service designation for a datapacket. In addition, the step of applying the predetermined global maskincludes masking all but a desired service type designation to generatepacket forwarding data corresponding to the desired service type.

According to another aspect of the embodiments, each CAM entry caninclude a predetermined number of service type bit locations, each bitlocation corresponding to a different service type designation. Eachservice type bit location can have a set state and not set state.

According to another aspect of the embodiments, a step of storing atleast one data value can include setting a predetermined service typebit for a plurality of entries to a set state to establish a defaultservice value for the data values of the entries.

According to another aspect of the embodiments, a step of storing atleast one data value can further include storing a priority forwardingvalue having the same plurality of matching field values as the defaultforwarding entry, the priority forwarding entry having less than allservice type bits with set states and a higher priority physical addresslocation.

According to another aspect of the embodiments, each CAM entry furtherincludes at least a destination field that indicates the destination ofa data packet.

According to another aspect of the embodiments, the method can furtherinclude receiving a delete command to delete an existing first data setentry. A delete command can indicate matching field values and at leastone service type. In response to a delete command, the method caninclude finding each first data set entry having fields that match thefields of the delete command. If the service type designations of theentry do not match all the fields of at least one service type of thedelete command, setting the matching service type designations to anon-set state. If all of the service type designations of the entrymatch at least one service type of the delete command, deleting theentry.

According to another aspect of the embodiments, the method can includereceiving a write command indicating matching field values and at leastone service type. The method can also include, in response to the writecommand, finding an entry having fields that match the matching fieldsof the write command. If the service type designations of the entry donot match the service type of the write command, setting the matchingservice type designation of the entry to a non-set state.

The present invention can also include a method for providing multipledata set values with a single set of globally maskable contentaddressable memory (CAM) entries. The method can include storing firstdata set values in the CAM entries, each data set value comprising aplurality of unique fields. In addition, the method can includeutilizing a CAM entry that stores a first data set value tosimultaneously store a proxy value for at least one other data set. Eachproxy value can represent a data set value that shares a subset of theunique fields of the first data set values.

According to one aspect of the embodiments, the method can includestoring associated data for each first data set value in a first randomaccess memory (RAM) section accessed with a first table offset addressvalue. In addition, the method can include storing associated data forother data set in a corresponding other RAM section accessed with otheroffset address values unique to each other data set.

According to one aspect of the embodiments, a method can include onetype of read function. Such a read function can include reading RAM dataaccording to the primary table address offset values and an index valuegenerated by searching the primary table CAM entries. According to oneaspect of the embodiments, a method can include another type of readfunction. Such a function can include reading RAM data according to oneof the aggregate address offset values and an index value generated bysearching proxy entries for an aggregation scheme.

Such an arrangement can enable rapid access of primary table associateddata, and can be useful in operations such as flow updating in responseto new packet data. Such a function can enable rapid access of aggregatetable associated data. Further, such a function can be useful whenupdating aggregation data in response to the expiration of a flow entry.According to one aspect of the embodiments, the method can include, inone type of data modify operation, modifying data read from a RAMlocation accessed by one of the address offset values and an index valueaccording to data from a newly arrived data packet. According to anotheraspect of the embodiments, a method can include in another type of datamodify operation, modifying data read from a RAM location accessed byone of the aggregate address offset values according to data read from aRAM location accessed by the primary table address offset value.

Such a feature can enable rapid modification of primary table associateddata. This function can be useful when updating primary table associateddata in response to new packet information. Such features can enablerapid modification of aggregate associated data. This can be useful whenupdating aggregate associated data in response to a deleted primarytable entry.

The present invention can also include a system for searching multipledata set values with a single content addressable memory (CAM) table.The system can include a CAM section having N globally maskable CAMentries, each CAM entry operable to store N data set values that indexto first set associated data, at least one second set associated data,or both first and second set associated data. The system can alsoinclude a control section having an address generator that generatesoffset addresses corresponding to a predetermined global mask value.Offset addresses can include a first data set offset address and aplurality of secondary set offset addresses each corresponding to adifferent set of associated data.

According to another aspect of the embodiments, an associated datasection can include a first section having no more than N entries thatstore first set associated data, and at least a second section having nomore than N entries for storing second set associated data.

According to another aspect of the embodiments, a control section can bean application specific integrated circuit formed on a differentintegrated circuit than the CAM section, a programmable logic deviceformed on a different integrated circuit than the CAM section, or aprocessor executing instructions.

Such an arrangement can allow a system to be formed with an “off theshelf” CAM device in conjunction with other circuit components.

According to another aspect of the embodiments, a control section can beformed in the same integrated circuit as the CAM section.

According to another aspect of the embodiments, each CAM entry caninclude a plurality of valid bits. A CAM section can include a globalmask register that generates global masks for searching the CAM entries,the global mask register including a global mask corresponding to eachvalid bit.

Such an arrangement can provide for a fast, reliable way ofdistinguishing between entries corresponding to different data sets.

According to another aspect of the embodiments, each CAM entry caninclude valid bits, each valid bit corresponding to one of a pluralityof schemes. The schemes can include a primary table and each of theaggregation schemes. In addition, the CAM section can include globalmask registers that generate global masks for searching the CAM entries.The global mask registers can include a global mask corresponding toeach scheme that masks all valid bits except the valid bit of thecorresponding scheme.

Such an arrangement can provide for a fast, reliable way ofdistinguishing between entries corresponding to a primary table, andentries corresponding to aggregate tables.

According to another aspect of the embodiments, a control section canfurther include an input parsing section that adds valid bits to areceived search key data to generate an internal search key applied tothe CAM registers

According to another aspect of the embodiments, a control section canfurther include an address modifier that generates a new address basedon an offset address from the address generator and an index value fromthe CAM section.

According to another aspect of the embodiments, the data set values thatindex to the first set associated data and both the first and second setassociated data comprises a primary table, where each data set valuedefines a network flow. In addition, data set values that index to thesecond set of associated data and both the first and second setassociated data comprises an aggregation table, where each data setvalue defines a network flow data aggregation scheme.

According to another aspect of the embodiments, data set values thatindex to the first set associated data can include forwardinginformation for a first type of packet transmission service. Inaddition, data set values that index to the second set of associateddata can comprise forwarding information for a second type of packettransmission service.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B show is a block diagram of a first embodiment of thepresent invention.

FIG. 2 is a block diagram of a second embodiment of the presentinvention.

FIGS. 3A and 3B are block diagrams showing the operation of theembodiment set forth in FIG. 2.

FIG. 4A is a block diagram showing a more detailed embodiment of thepresent invention. FIG. 4B is a table showing variables that will beused in the description of the embodiments.

FIG. 4C is a block diagram showing a conventional example of flow dataaggregation. FIG. 4D is a block diagram showing aggregation of the sameflow data as FIG. 4C according to an embodiment of the presentinvention.

FIG. 5 is a block schematic diagram of a system according to anembodiment of the present invention.

FIG. 6A is a diagram showing a sample format of a key received by theembodiment of FIG. 5. FIG. 6B is a diagram showing a sample format foran internal search generated by the embodiment of FIG. 5. FIG. 6C is adiagram showing a sample format for a global mask generated by theembodiment of FIG. 5.

FIG. 7 is a block diagram of a CAM section that can be used in theembodiment of FIG. 4.

FIG. 8 is a block schematic diagram of a control section that can beused in the embodiment of FIG. 4.

FIG. 9A is a block schematic diagram of a logic section that can be usedin the control section shown in FIG. 8. FIGS. 9B to 9F show theoperation of the logic section of FIG. 9A during various functionsaccording to an embodiment of the present invention.

FIG. 10 is a flow diagram showing a look-up function according to oneembodiment.

FIGS. 11A and 11B show a flow diagram of an entry deletion functionaccording to one embodiment.

FIGS. 12A and 12B show a flow diagram of an entry insertion functionaccording to one embodiment.

FIG. 13 is a flow diagram of an application function according to oneembodiment.

FIG. 14 is a diagram showing how each higher level function can utilizelower level functions to provide data aggregation operations.

FIG. 15 is a table illustrating look-up rate requirements for dataaggregation based on different line speeds.

FIG. 16 is a table illustrating the number of first level (device)functions needed for each second level (routine) function according toan embodiment of the present invention.

FIG. 17 is a table illustrating how the second level functions can meetthe search and data access rates needed for higher line speeds.

FIGS. 18A and 18B are block diagrams of alternate embodiments of thepresent invention.

FIG. 19 is a block diagram of another embodiment of the presentinvention.

FIG. 20A is a block diagram showing a conventional example of forwardingdata sets. FIG. 20B is a block diagram showing virtualization of thesame forwarding data sets as FIG. 20A according to an embodiment of thepresent invention.

FIG. 21 is a flow diagram of an entry deletion function according toanother embodiment.

FIG. 22 is a flow diagram of an entry insertion function according toanother embodiment.

FIG. 23 is a table illustrating flow data of a conventional primarytable.

FIG. 24 is a diagram illustrating conventional approaches to flow dataaggregation.

FIG. 25 is a block diagram showing a first conventional system foraggregating data that utilizes hashing.

FIG. 26 is a block diagram showing a second conventional system foraggregating data.

DETAILED DESCRIPTION

Various embodiments of the present invention will now be described indetail with reference to a number of drawings. The embodiments include asystem and method that utilizes a ternary content addressable memory(TCAM) having a global masking capability to manage multiple data tableswith less TCAM resources than conventional approaches.

It is understood that data “table” or data “set” as described herein,can include sets of multiple data values, where each data value iscomposed of multiple data fields. Data values having the same datafields can be considered to be included in the same data set. Accordingto the embodiments described, two or more data sets can be formed in thesame set of physical TCAM entries by utilizing one physical entry torepresent a data value in two different sets. A physical entry can bethe actual physical circuitry on a TCAM device for storing a data value(or in this case multiple data values of different sets) to thereby forman entry. Each such physical entry can be accessed according to aphysical address.

In this way, one set of TCAM entries can “virtually” represent multipledata sets, but not have to include one physical entry for each value ofeach different data set. In the embodiments described, a global maskableTCAM can have entries with search fields (keys) corresponding tomultiple data tables. Differentiation between data sets can beaccomplished according to global masking of data fields. Optionally,each entry can include table identifier bits to further identify thoseentries belonging to multiple data sets.

System and Device—Generally

FIGS. 1A and 1B show a method and system according to a firstembodiment. A first embodiment 100 can include a number of globallymaskable CAM entries 102. It is understood that these CAM entries 102are physical entries of a CAM device. CAM entries 102 are represented inFIGS. 1A and 1B by entries 102-0 and 102-1, but it is understood thatCAM entries 102 can include numerous such entries, all of which can becompared to a search key essentially simultaneously.

CAM entries 102 are “globally” maskable according to a global mask 104.A global mask 104 can all mask entries 102 from a compare (e.g., search)operation by bit location, or multiple bit locations. This is incontrast to conventional ternary masking, which enables bit-by-bitmasking on a per entry basis.

In such an arrangement, selective global masks can be used generatemultiple match indications from a single entry, with each differentmatch indication corresponding to a different data set.

CAM entries (102-0 and 102-1) can each store a data value composed ofmultiple fields. In the example of FIGS. 1A and 1B, each CAM entry(102-0 and 102-1) stores four field values F1, F2, F3 and F4. While CAMentries 102-0 and 102-1 have common values for fields F1 and F2 (A andB, respectively), CAM entry 102-0 differs from CAM entry 102-1 in fieldsF3 and F4.

The particular example of FIGS. 1A and 1B shows the virtualization oftwo data sets in the same set of CAM entries. A first data set includesvalues having fields F1/F2/F3/F4. A second data set includes only fieldsF1/F2. FIGS. 1A and 1B show how global masking can differentiate betweendifferent data set values stored in a same CAM entry.

In FIG. 1A, a key value is applied having values A/B/C/D. According toconventional CAM matching functions, the key value is compared to storedCAM entry values to determine if a match exists. In FIG. 1A, the keyvalues match CAM entry 102-0, and thus generate a match indicationMATCH0.

A match indication MATCH0 can be utilized to access associated datacorresponding to a first data set. In the particular example of FIG. 1A,match indication MATCH0 is combined with a set value SET0 to access afirst associated data section 108-0.

In FIG. 1B, a key value is applied having values A/B/E/G. Ordinarily,such a key value would match CAM entry 102-1. However, in the exampleshown, global mask 104 masks fields F3 and F4 from a compare operation.As a result, a match occurs in a highest priority CAM entry between102-0 and 102-1 (assumed in this case to be CAM entry 102-0). Thus, CAMentry 102-0 generates another match indication MATCH0. However, such amatch indication MATCH0 can be utilized to access associated datacorresponding to a second data set.

In the particular example of FIG. 1B, match indication MATCH0 iscombined with a set value SET1 to access a second associated datasection 108-1.

It is understood that fields (e.g., F1 to F4) can be of arbitrary bitsize. That is such fields can represent multi-bit criteria (e.g.,network flow descriptor data) or single bit criteria (e.g., valid bitdata), or some combination thereof.

In this way, a single instance of a globally maskable CAM array canvirtually present multiple data sets for search.

System and Device—Data Aggregation

In the embodiments described, a global maskable TCAM can have physicalentries with search fields (keys) corresponding to multiple data tables.Differentiation between data sets can be accomplished according toglobal masking of data fields. Optionally, each entry can include tableidentifier bits to further identify those entries belonging to multipledata sets. Associated data for both primary and secondary tables can bestored in a different type of memory (e.g., DRAM or SRAM), preferablyone having a lower cost per bit.

A system for providing data aggregation according to another embodimentis set forth in a block diagram in FIG. 2, and designated by the generalreference character 200. A system 200 can include a CAM portion 202 andan associated data portion 204. A CAM portion 202 can be formed fromentries of one more CAM devices having a global masking capability.

As but one example, the Ayama 10000 Network Search Engine, manufacturedby Cypress Semiconductor Corporation, of San Jose, Calif. includes 16pairs of global mask registers, each pair being 144-bits wide.

Each entry of CAM portion 202 can store key searchable data for aprimary table. In this case, such data can be the data of relevant flowdescriptor fields. By use of global masking and aggregation defining bitvalues, such entries can also serve as key data for one or moresecondary tables, or as keys for both primary and secondary tables.Thus, a CAM portion 202 can be conceptualized as including a single setof physical CAM entries that store multiple data sets, with at least onephysical entry storing different data values of different data sets.

As a result, unlike the conventional arrangement of FIG. 26, a system200 can include one table (primary table 206) for key data in CAMportion 202, instead of multiple tables (primary and secondary tables).Accordingly, for a system providing “m” data aggregation schemes, afirst embodiment 200 can provide about a 1/(m+1) reduction in needed CAMentries, as compared to the conventional approach of FIG. 26.

Entries of primary table 206 can include key portions 206-0 that candefine unique primary data values. In addition, and unlike conventionalapproaches, each entry can also include a table identifier portion206-1. A table identifier portion 206-1 can indicate if a CAM entrycorresponds to a primary table or to any of the multiple secondarytables.

An associated data portion 204 can store associated data for bothprimary and secondary tables. Thus, in the example of FIG. 2, associateddata portion 204 can include primary data area 208-0 for storingassociated data corresponding to a primary table (e.g., data associatedwith individual flows), and aggregate data areas 208-1 to 208-m, forstoring associated data corresponding to “m” secondary tables (e.g.,aggregate data).

An associated data portion 204 is preferably formed from RAM devices,even more preferably from DRAM, which may provide an advantageously lowcost per bit.

In this way a system 200 can provide high look-up rates (CAM speed) toassociated data in a primary table and multiple secondary tablestotaling (m+1)*N RAM entries, with only about N CAM entries.

Referring now to FIGS. 3A and 3B, two block diagrams are shownrepresenting how primary and aggregate data can be accessed by utilizingglobal masking within table identifier portion 206-1. FIGS. 3A and 3Bshow the structure of FIG. 2, and in addition, shows the application ofa search key 300 according to a global mask 302.

In FIG. 3A, a global mask 302 is configured as a primary table mask.Thus, in response to a “hit”, a modify operation 304-0 can access anentry in primary data area 208-0. A primary table mask can mask out bitvalues designating secondary tables, and thus identify only thoseentries representing primary table keys values. In contrast, in FIG. 3B,a global mask 302 is configured as an aggregate table mask. A secondarytable mask can mask out a bit value (or bit values) designating aprimary table, as well as those for non-relevant secondary tables, andthus identify for search the key values of one secondary table. Thus, inFIG. 3B in response to a “hit”, a modify operation 304-1 can access anentry in an aggregate data area 208-i.

In a preferable arrangement of FIG. 3B, a key value 300 is not a keyrelated to an incoming packet, but rather a key generated in response toaggregation related command/instruction.

In this way, the same physical CAM entry can be switched betweenrepresenting primary data to representing aggregate data by setting one(or possibly more) bits in a table identifier portion 206-1. Inaddition, searches can apply a global mask that distinguishes betweensuch values.

A more detailed embodiment of the present invention is set forth in FIG.4A, and designated by the general reference character 400. Theembodiment of FIG. 4A includes some of the same general items as FIG. 2.Thus, like items will be referred to by the same reference character asFIG. 2, but with the first digit being a “4” instead of a “2”.

FIG. 4A shows a CAM portion 402 that includes “N” physical entries (E₁to E_(N)), each having a key portion 406-0 and table ID portion 406-1.In the particular example shown, a key portion 406-0 can store a keyformed from “K” fields (F₁ to F_(k)) and a table ID portion 406-1 formedfrom “m”+1 valid bits. Further, of the m+1 valid bits, one valid bit V₀can identify a primary table value, while valid bits V₁ to V_(m) caneach identify a different aggregation scheme. Thus, searches directed toa primary table value can use a global mask that masks out valid bits V₁to V_(m). Conversely, a search directed to a particular aggregationscheme indicated by valid bit “V_(i)” will use a global mask that masksout valid bit V₀, and all other valid bits except V_(i).

Within associated data portion 404, each data area (408-0 to 408-m) canstart at an address indicated by an address pointer (represented byitems 410-0 and 410-m). In such an arrangement, associated data can beaccessed by adding an address pointer value to an index value generatedby a CAM portion 402, as will be described in more detail below.

In the very particular example shown, a CAM portion 402 can have “N”entries and each data area (408-0 to 408-m) can have N entries.

The arrangement of FIG. 4A shows a number of variables that will bereused in descriptions below, and so are listed in Table form in FIG.4B.

To better understand data aggregation according to the presentinvention, one very particular example showing the aggregation of flowsis shown in FIGS. 4C and 4D.

FIG. 4C shows a conventional approach like that shown in FIG. 26 forstoring flow values in a CAM section 456 and associated data in a RAMsection 458. Thus, CAM section 456 includes a primary table 456-0, alongwith multiple secondary tables 456-1 to 456-m. In the very particularexample of FIG. 4C, primary table 456-0 includes four flows, defined byan IP source address (S_IP), an IP destination address (D_IP), aprotocol type (Proto), a Source Port (SPort) and a Destination Port(DPort).

Two of the secondary table aggregation schemes are also shown in FIG.4C. For secondary table 456-1, data is aggregated based on fields S_IP,D_IP and Proto. For secondary table 456-m, data is aggregated based onthe S_IP field.

A RAM section 458 can include associated data 458-0 for the primarytable, as well as associated data for secondary tables 458-1 to 458-m.In the very particular example of FIGS. 4C and 4D, associated data caninclude two fields: number of packets (# of Pkts) and number of bytes(Bytes).

FIG. 4D shows how the data illustrated in the conventional case of FIG.4C can be more compactly fit into a single CAM table. Thus, FIG. 4Dshows only one table 476-0 in CAM section 476 containing the four flowentries. Each defines a current flow, as shown by valid bit “V0” beingset to “1”. In addition, first, second and fourth entries can beconsidered “proxy” entries, and these entries can point to associateddata section 478-1. This is shown by valid bit “V1” being set to “1” forthese entries. Finally, the first and second entries also serves as aproxies for a last associated data section 478-m as shown by valid bitVm being set to “1” for these entries. Thus, single physical entriesstore multiple data values (e.g., act as “proxies”).

A RAM portion 478 of FIG. 4D can be essentially the same as that of FIG.4C.

Of course, FIG. 4D represents but one possible application of thepresent invention.

The above embodiments and examples thus clearly show that, unlikeconventional arrangements (like that of FIG. 26) different CAM entriesfor primary and secondary tables are not needed, reducing the amount ofoverall CAM required, while at the same time supporting a large numberof secondary tables for aggregation data.

Another detailed embodiment of the present invention is set forth inFIG. 5, and designated by the general reference character 500. A system500 can correspond to any of the above-described embodiments. A system500 can include a search device 502 and one or more RAM devices thatform a RAM section 504. A search device 502 can include a controlsection 506 and a ternary CAM (TCAM) section 508 that provides searchand related operations. A CAM section 508 can include maskable CAMentries for executing various CAM related functions, including globallymaskable compare operations with applied search key values, as well aswrites to and reads from such CAM entries. A CAM section 508 can alsoinclude global mask registers 510 for storing global masks, as describedabove.

A control section 506 can receive key values (Key) and commands values(Cmd). In response to such values, a control section can provide dataand control signals (e.g., commands) to both CAM section 508 and RAMsection 504. More particularly, a control section 506 can provide anumber of device functions detailed below. Such functions can includearithmetic or other operations on received data values. Further, suchfunctions can be executed in predetermined orders to provide higherlevel routines for use at an application level. This too will bedescribed in more detail below.

A control section 506 may be formed in a separate integrated circuitthan a CAM section 508. That is, a control section 506 may be anapplication specific integrated circuit (ASIC), or may be implemented byprogrammable logic, such as an FPGA, as but two examples. Still further,a control section 506 can include registers manipulated by a processoraccording to predetermined instructions.

Alternatively, a control section can be included within the samesubstrate as a CAM section 508.

A RAM section 504 can preferably be formed from “off the shelf”synchronous dynamic random access memories (SDRAMs). However, alternatememory types could be utilized, including but not limited to other typesof DRAMs, static RAM (SRAMs), or other fast access RAMs.

First Level Functions.

The following portions of the description will refer to particularfunctions related to devices and methods of the present invention. Suchfunctions will be described according to a particular syntax, asdetailed below.

The functions include, but are not limited to, a TCAM search (NSearch),a TCAM write (NWrite), a TCAM read (NRead), a RAM write (DWrite), and aRAM read (DRead).

NSearch.

An NSearch(K,i) function can execute a search on CAM entries accordingto a key value “K” and a scheme value “i”. Further, a scheme value “i”can select a particular global mask utilized in the search. An exampleof such an arrangement is shown in FIGS. 6A to 6C.

FIG. 6A shows a sample key value “K” that includes a number of fields F₁to F_(k).

FIG. 6B shows one example of how an NSearch(K,i) function can generate amodified key value K′ based on key value K and scheme value “i”. Inparticular, a new key value K′ can include the key value K along withvalid bits V₀ to V_(m). With respect to the valid bits (V₀ to V_(m)),the i^(th) valid bit can be set to one value (in this example, 1), whileall remaining valid bits are set to another value (in this example, 0).Each valid bit can indicate a particular scheme (e.g., primary table orone of the aggregate data tables). Thus, the setting of the valid bitdirects the TCAM search to a particular scheme.

FIG. 6C shows a global mask corresponding to an NSearch(K,i) function. Aglobal mask can be configured prior to a search operation, or generatedby a search operation. A global mask can include fields and valid bitlocations corresponding to the generated key value K′. As shown, aglobal mask can include non-masking values (in this case, 1) for the bitlocations of the key fields. Further, with respect to the valid bitlocations, the i^(th) masking bit can be set to a non-masking value,while all remaining valid bits can be set to masking value (in thisexample, 0).

In this way, a global mask can indicate one hit value for the indicatedscheme, and mask out any other schemes the same CAM entry mightcorrespond to.

An NSearch command will be described herein according to the followingsyntax:

(hit, index)=NSearch(K,i)

Inputs: K=Search Key; i=scheme (i^(th) global mask in CAM)

Outputs: hit=indicates whether search yielded a hit or miss;

-   -   index=location of CAM entry corresponding to search hit.        NWrite.

An NWrite(address,data) function can write data to a CAM location. Datawritten to a CAM location can be maskable according to a global maskvalue.

An NWrite command will be described herein according to the followingsyntax:

NWrite(address,data)

Inputs: address=CAM address where data is to be written;

-   -   data=data to be written to the CAM location.        NRead.

An NRead(address) function can read data from a CAM location.

An NRead command will be described herein according to the followingsyntax:

data=NRead(address)

Input: address=CAM address from which data is to be read;

Output: data=data read from the CAM location.

DWrite.

A DWrite(address,data) function can write data to a RAM location.

A DWrite command will be described herein according to the followingsyntax:

DWrite(address,data)

Inputs: address=RAM address where data is to be written;

-   -   data=data to be written to the RAM location.        DRead.

A DRead(address) function can read data from a RAM location.

A DRead command will be described herein according to the followingsyntax:

data=DRead(address)

Input: address=RAM address where data is to be written;

Output: data=data read from the RAM location.

System Component Examples.

Referring now to FIG. 7, a block schematic diagram is shown illustratingone particular example of a CAM section 700 that can be used as a CAMsection like that shown as 508 in FIG. 5. A CAM section 700 can includea CAM cell array 702, an address decoder 704, a priority encoder(PE)/logic section 706, a global mask register array 708, and drivercircuits 710. A CAM section 700 can also include a CAM data bus(CAM_BUS) 712 and a CAM result bus 714 (CAM_RES). A CAM section 700 canreceive control data (e.g., command and clocks) by way of a controlinput 716.

Having described the general structure of a CAM section 700, theoperation of the CAM section 700 during the various functions above willnow be described.

In an NSEARCH operation, a global mask value can be selected from globalmask register array 708 according to a scheme value “i”. A key valuealong with valid bits can then be applied to CAM cell array 702according to the selected global mask value by driver circuit 710. Ahighest priority matching entry can be encoded by PE/logic section 706to provide an index value, along with other information, such as a HITindication, or the like. A HIT indication can be a flag or other datavalue that can indicate whether the search generated a hit, miss, ormultiple hits, as but a few examples. An index and hit indication can beoutput on CAM result bus 714.

In an NWrite and NRead operation, an address value corresponding to aCAM entry can first be placed on CAM data bus 712. Such an address canbe decoded in address decoder 704 to select a particular CAM entry. Inan NWrite operation, write data can then be placed on bus 712 andwritten into a CAM entry. In an NRead operation, read data can then beoutput on bus 712.

Of course, the CAM section 700 of FIG. 7 is but one very particularexample of CAM section, and so should not be construed as limiting theinvention thereto.

Referring now to FIG. 8, a block schematic diagram is shown illustratingone particular example of a control section 800 that can be used as thatshown as 506 in FIG. 5. A control section 800 can include an input parsesection 802, a command decoder 804, a logic section 806 and a RAMinterface 808. An input parse section 802 can arrange incoming fieldvalues to appropriate bit locations for application to a CAM section. Inaddition, and unlike conventional arrangements, an incoming parsesection can generate valid bits (V0 to Vm) for a key value. In one veryparticular arrangement, such an operation can be accomplished by way ofa decoder that decodes a value “i” into a “one-hot” set of valid bits(V0 to Vm). Still further, in the case of erase operations, all validbits can be set to zero. Of course, such a decoding operation could beexecuted within a CAM section closer to driver circuits.

A resulting key value from input parse section 802 can be placed ontoCAM data bus CAM_BUS.

A command decoder 804 can decode an incoming command (Cmd) to generateCAM control signals (CAM_CTRL) for a CAM section, logic control signals(LOG_CTRL) for logic section 806, and RAM interface control signals(RAMI/F_CTRL) for RAM interface 808. Commands can include, but are by nomeans limited to, those described above: NSearch, NWrite, NRead, DWrite,and DRead.

A RAM interface 808 can access a RAM section by generating RAM addresssignals on an address bus (RAM_ADD) and control signals on control lines(RAM_CTRL). RAM data can be output and input by way of bi-directionaldata bus (RAM_DQ). As but one particular example, a RAM interface 808can be a synchronous DRAM interface. Of course, the particular interfacecan vary according to the structure of memory section.

A logic section 806 can receive input data values from a CAM data busCAM_BUS, a CAM result bus CAM_RES, and a RAM interface 808. In responseto logic control signals LOG_CTRL, logic section 806 can provide datapaths and operations according to the particular decoded command. Thesewill be described in greater detail below.

Referring now to FIGS. 9A to 9F, a block schematic diagram is shownillustrating a portion of a logic section 900, like that shown as 806 inFIG. 4. Logic section portion 900 can include an address pointergenerator 902, adders 904 and 906, multiplexers (MUXs) 908 to 912, anddata latches 914 to 918. Logic section 900 can receive data by way ofCAM data bus (CAM_BUS), CAM result bus (CAM_RES), and RAM data bus,(RAM_DB).

An address pointer generator 902 can receive a value “i” (also utilizedto encode valid bits and/or select a global mask). In response to such avalue, address pointer generator 902 can output an address pointerassociated with a primary table (e.g., AddPtr[0]) or an address pointerfor one of the aggregated data tables (e.g., AddPtr[1] to AddPtr[m]). Anaddress pointer generator 902 may preferably included address registersselectable according to a value “i”. Of course, addresses could begenerated by alternate means, including an arithmetic operation and/oraccessing “hard wired” values.

An adder 904 can provide arithmetic functions for modifying values readfrom a RAM section, by newly provided data. Adder 906 can add an indexvalue received on CAM result bus CAM_RES with an address pointer valueto generate an address for a RAM section. Such operations will bedescribed in more detail below.

Having described the general structure of a logic section portion 900,the operation of such a section during the various functions above willnow be described.

FIG. 9B shows the operation of the logic section portion 900 during aparticular function: DRead (AddPtr[i]+index). This function can read adata value at a RAM address generated by a search command to CAMsection. In such an operation, a value “i” can be applied to addresspointer generator 902, to generate an address pointer value AddPtr[i].In addition, a search operation can generate an index value “index” onCAM result bus CAM_RES. Adder 906 can add these values to generate a RAMaddress AddPtr[i]+index. Such an address can be applied to RAM sectionby way of latch 918. In response to such an address, a RAM section canprovide read data OLD_DATA. One skilled in the art would recognize thatan adder 906 that generates an address value need not be an arithmeticadder, but may simply combine higher order address pointer bits withlower order index bits.

FIG. 9C shows the operation of the logic section portion 900 duringanother function: WRITE_DATA=OLD_DATA+NEW_DATA. This operation canmodify data read from a RAM section with newly arrived data. In such anoperation, data previously read from a RAM section (OLD_DATA) can beadded to data provided on CAM data bus (NEW_DATA) within adder 904. Aresulting value (WRITE_DATA) can be stored in latch 914.

FIG. 9D shows a third operation of the logic section portion 900: DWrite(AddPtr[i]+index, WRITE_DATA). Such an operation can write modified databack into a RAM location. As shown in FIG. 9D, this operation can outputmodified data (WRITE_DATA) and the address generated as shown in FIG.9B. Thus, an address (AddPtr[i]+index) can be a value previouslygenerated and stored in latch 918.

The above-described functions can enable higher level operations, aswill be described below.

In addition to the above functions, a logic section portion 900 can alsoprovide conventional access to a RAM section. FIG. 9E shows an operationDWrite (NEW_ADD, NEW_DATA) that can write data to a RAM location. In theoperation, an address value NEW_ADD from CAM data bus CAM_BUS can beapplied by way of MUX 912. This can be followed by a write data valueNEW_DATA applied by way of MUX 908. In a similar fashion, FIG. 9F showsan operation DRead (NEW_ADD), which can read a data value from a RAMaddress.

Second Level Functions—Data Aggregation.

Having described first level device functions and exemplary components,various routines utilizing such functions will now be described withreference to FIGS. 10 to 12B.

GLookUP

FIG. 10 shows one example of an entry look-up function “GLookUp”according to one embodiment. A GLookUp function can be describedaccording to the following syntax:

GLookUp(K, i, NEW_DATA)

Inputs: K=Search Key; i=scheme (i^(th) global mask in CAM);

-   -   NEW_DATA=associated data of a packet.        Referring still to FIG. 10, a GLookUp function is designated by        the general reference character 1000 and can include executing a        search in a CAM section with a key K and scheme value i (1002).        As shown in the figure, this can include performing a first        level function NSearch to acquire output values “hit” and        “index”.

A function can then determine if a matching entry has been found fromthe search (1004). This can include examining a “hit” output value, anddetermining if it indicates a hit (hit=HIT).

If the search operation yielded a matching entry (Y branch from 1004),then an entry exists in the CAM section that matches the new key K.Consequently, the appropriate associated data scheme (S_(i)) can beupdated based on the value of “i” (1006). In the particular example ofFIG. 10, this can include reading “old” data from the appropriate RAMlocation utilizing the first level functionOLD_DATA=DRead(AddPtr[i]+index), updating such data with the operationWRITE_DATA=OLD_DATA+NEW_DATA, and then writing such modified data backinto the same RAM location with the function DWrite(AddPtr[i]+index,WRITE_DATA). Following such a write operation, the GLookUp function canend (1008).

If the search operation does not yield a matching entry (N branch from1004), then no entry exists for the data, and a new (or proxy) entrymust be created within CAM section with corresponding data being writteninto a RAM section (1010). In the example of FIG. 10, this can beaccomplished with another second level function GInsert(K, i, NEW_DATA),described in more detail below. Following such an insert operation, theGLookUp function can end (1008).

In this way, a CAM section can be searched to determine if an entryexists that represents a particular scheme (e.g., primary table oraggregate data table). If such an entry exists, the associated data canbe updated. If such an entry does not exist, it is created utilizing anexisting entry (i.e., creating a proxy) or writing a new entry into aCAM section.

GDelete.

FIGS. 11A and 11B shows one example of a CAM entry delete function“GDelete”. A GDelete function can be described according to thefollowing syntax:

GDelete(K, i)

Inputs: K=Search Key; i=scheme (i^(th) global mask in CAM).

Referring to FIG. 11A, a first portion of a GDelete function isdesignated by the general reference character 1100 and can includeexecuting a search in a CAM section with a key K and scheme value i(1102). As shown in the figure, this can include performing a firstlevel function NSearch to acquire output values “hit” and “index”.

The function 1100 can then determine if a matching entry has been foundfrom the search (1104). This can include examining a “hit” output value,and determining if it indicates a hit (hit=HIT).

If the search operation does not yield a matching entry (N branch from1104), then no entry exists for the key value. This indicates an errorcondition (1106). As a result, an error indication can be returned to arequesting device.

If the search operation yields a matching entry (Y branch from 1104),then an entry exists in the CAM section that needs to be deleted, and ascheme value “i” can be checked to determine if the entry for deletionexists in the primary table or is in one of the secondary (e.g.,aggregate) tables (1108). In the particular example of FIG. 11A, it isassumed that a value of i=0 indicates the primary table, while valuesi>0 each represent a different secondary table. Thus, step 1108 caninclude determining if the value i is zero (i==0).

If the to-be-deleted entry exists in a secondary table (N branch from1108), then a secondary table entry can be “deleted”. It will berecalled, that a same CAM entry can represent both a primary andsecondary tables according to valid bit data. Thus, “deletion” of asecond table entry can include setting valid bit for the secondary tableto a non-active value (in this case 0) (1110). In the example shown,such an operation can include setting the appropriate valid bit to zerobased on the received value of “i”: Set Vld_bit[i]=0. A new key valuecan then be written into the CAM entry with such a changed valid bitvalue utilizing the first level CAM write function: NWrite(index,Vld_bit[i]). This function can be a “masked” write, that only writes the“0” value to the valid bit location corresponding to the “i” value.

Depending upon the particular application, a GDelete function can alsoinclude exporting the associated for the deleted secondary table entry(1112). In the example shown, this can include reading out theassociated data with a first level function DATA=DRead(AddPtr[i]+index).Such data can then be output with a flag: return(0, DATA). A GDeletefunction can then end (1114).

Referring still to FIG. 11A, if the to-be-deleted entry exists in theprimary table (Y branch from 1108), then the appropriate primary tableentry can be “deleted”. As but one example, this can represent aparticular flow value being aggregated (e.g., flow has timed out,terminated, etc.). Consequently, the latest associated data for theprimary table entry can be read out from a RAM section (1116). In theexample shown this can be accomplished with a first level function:NEW_DATA=DRead(AddPtr[0]+index).

Referring now to FIG. 11B, deletion of a primary entry can continue byupdating all current aggregation schemes according to the newly readdata from the deleted primary entry (1118). In the particular example ofFIG. 11B, this can include aggregating data for all secondary tables.Assuming that a value i=0 corresponds to a primary table, this caninclude aggregating for all values i>0 (1120). This is shown in FIG. 11Bby the conditional operation “for j=1 to m”.

Aggregating for a deleted primary table entry can first includesearching for an existing primary table that currently represents theaggregation scheme (1122). This is shown in FIG. 11B by execution of afirst level search function aimed at the current secondary table: (hit2,index2)=NSearch(K, j). It is understood that such a step can include themasking out of non-relevant fields.

The function 1100 can then determine if a matching entry has been foundfrom the secondary table search (1124). This can include examining a“hit2” output value, and determining if it indicates a hit (hit2=HIT).Such a search result can indicate whether another entry already existsthat represents the aggregate data (a “proxy” exists), or whether theto-be-deleted entry must be converted into a proxy entry (an entry thatcan represent one or more secondary tables).

If the search indicates that an entry exists for the aggregation scheme(Y branch from 1124), then a proxy exists. Thus, correspondingassociated data for the proxy entry is updated with the associated dataof the deleted primary entry (1126). In the example of FIG. 11B, thiscan be accomplished with first level functions. In particular, functionOLD_DATA=DRead(AddPtr[j]+index2) can retrieve associated data for thesecondary table by searching with the proxy entry values. Such read datacan be added with the deleted entry data with the operationWRITE_DATA=OLD_DATA+NEW_DATA. Such data can then be written back asupdated associated data for the secondary table by writing based on theproxy entry values: DWrite(AddPrt[j]+index2, WRITE_DATA). Following suchoperations, a function can proceed to a next aggregation scheme (1128).

If the search indicates that an entry does not exist for the aggregationscheme (N branch from 1124), then the primary entry being deleted isconverted into a proxy entry for the secondary table (1130). In theexample of FIG. 11B, this can be accomplished with first levelfunctions. In particular, a valid bit for the current secondary tablecan be set to an active value Set Vld_bit[j]=1. The valid bit for theprimary entry is then changed with a masked write operationNWrite(index, Vld_bit[j]). Associated data for the new secondary tablecan then be created by writing to associate data of the primary entry tothe appropriate RAM location DWrite(AddPtr[j]+index, NEW_DATA).Following such operations, a function can proceed to a next aggregationscheme (1128).

Once all aggregation schemes have been checked (End of Loop branch from1120), a GDelete function can return a predetermined indication showingthe operation has been completed (1132). This is represented in FIG. 11Bby the operation “return (1,0)”.

In this way, when a primary table entry is deleted, its associated datais aggregated to an existing secondary table according to an alreadyexisting proxy entry, if possible. If not possible, the entry isconverted into a proxy entry for non-represented aggregation schemes.

GInsert.

FIGS. 12A and 12B shows one example of a CAM entry insert function“GInsert”. It will be recalled that such a function can be included inthe GLookUp function described with reference to FIG. 10. A GInsertfunction can be described according to the following syntax:

Success=GInsert (K, i=0, NEW_DATA)

Inputs: K=Search Key; i=0 (primary table);

-   -   NEW_DATA=associated data

Outputs: Success=a flag indicating success/failure of insertion of entry

Referring to FIG. 12A, a GInsert function can first check for CAMentries that match a value to be inserted, but correspond to one or moresecondary tables (1202). This is shown in FIG. 12A by a first levelsearch operation (hit, index)=NSearch(K,j) (1204) searching CAM entriesbased on valid bit locations 1 to m (1205).

Following each such search, a hit indication can be checked to see of anentry exists that matches (1206). If no entry exists for the currentsecondary table, a search based on a next secondary table can takeplace. This is shown by the Next j step (1208).

If all possible secondary table searches yield no matches (“End of Loop”path from 1205), then no entry currently exists that matches the newlyarrived data. A CAM section can then be examined for an available entry(1210). In the particular example of FIG. 12A, this can includegenerating a new search key K′having all valid bits set to zero.Further, a global mask GM′can be generated having all valid bitlocations set to non-mask states (in this case all 1's). A first levelfunction search can then be performed with such values (hit2,index2)=NSearch(K′,GM′).

In this way, a CAM section can be searched for entries not belonging toany scheme (all valid bits=0).

A resulting search result can then be examined for a HIT result (1212).

Referring now to FIG. 12B, if no hit results (N branch from 1212), thenthere is no available entry in a CAM section. A GInsert function canreturn an unsuccessful result value return(0) (1214). The GInsertfunction can then end (1216).

If a hit does result (Y branch from 1212), an entry is available in theCAM section, and the newly arrived data can be written to such alocation (1218). In the particular example of FIG. 12B, this can includesetting a primary scheme valid bit to an active value Set Vld_bit[0]=1.A key value K′can then be generated that includes the new valid bitvalues K′=[K, Vld_bit[0]). Such a value can then be written to thelocation indicated by the search: NWrite (index2, K′).

Having created a new primary table entry, associated data for the entrycan be written to the appropriate location in a RAM section (1220). Inthe example of FIG. 12B, this can be accomplished with a first level RAMwrite function DWrite (AddPtr[0]+index2, NEW_DATA). A GInsert functioncan then return an indication that the function is complete (1222).

Referring back to FIG. 12A, if an initial search for the to-be-inserteddata indicates a hit (Y branch of 1206), the valid bits for the matchingentry can be examined. In the example of FIG. 12A, this can includeutilizing a first level RAM read function Vld_Bit=NRead(index).

Once the valid bits for the matching entry have been read. The bits canbe examined to determine if the matching entry represents a conflict. Inthe arrangement shown, a conflict can occur when a CAM entry serves as aproxy for multiple entries (1226). That is, will the insertion of thenew entry value cause a conflict with an existing aggregation scheme. Inthe example of FIG. 12A, this can include examining each valid bit tosee if it is active (1228 and 1230). If a valid bit location is notactive (N branch from 1230), a next bit value can be examined (1232). Ifa valid bit location is active (Y branch from 1230), it can be checkedto determine if it raises a conflict (1234).

Checking for a conflict can include confirming that writing inserteddata (K) will not invalidate a current proxy entry. For example, thefield data representing an aggregation scheme corresponding to theactive valid bit can be checked to ensure it matches that of theto-be-inserted primary entry data. If the data fields match, there is noconflict for the particular proxy entry. If the data fields do notmatch, there is a conflict, and the primary entry data is not suitablefor the location.

If a bit location presents a conflict (Y branch of 1234), a next bitlocation can be checked (1232). If all active bit locations foundpresent a conflict, a next second location can be examined by proceedingto a next “j” value (1236).

Referring now to FIG. 12B, if a bit location does not present a conflict(N branch of 1234), the entry can be “inserted” at the matching CAMlocation (1236). In the particular example of FIG. 12B, this can includesetting a primary scheme valid bit to an active value Set Vld_bit[0]=1.A key value K′can then be generated that includes the new valid bitvalues K′=[K, Vld_bit[0]). Such a value can then be written to thelocation indicated by the search: NWrite (index, K′).

Having written the new data value in the existing CAM entry, associateddata for the entry can be written to the appropriate location in a RAMsection (1238). In the example of FIG. 12B, this can be accomplishedwith a first level RAM write function DWrite (AddPtr[0]+index,NEW_DATA). A GInsert function can then return an indication that thefunction was successful (1240).

A function can then break out of examining valid bits (1242), and thefunction can end.

In this way, when a new entry is to be added to a database, an entirelynew entry can be added, or an existing entry can be converted (bysetting valid bits) to function as a “proxy” for such an entry.

Third Level Functions.

Having described first level device functions, and higher second levelfunctions that can utilize first level functions, a higher (third) levelfunction (referred to herein as an application) will now be described.

Referring now to FIG. 13, an application 1300 can execute various secondlevel functions according to particular events. One such application caninclude the deletion of a primary table entry (1302). As but a fewexamples, such an event can occur when an existing flow has been idletoo long, has exceeded a particular duration, or the primary table isfull, necessitating deletion based on more aggressive criteria. In suchan event, an application 1300 can execute the GDelete function (1304),utilizing a key value K to identify the entry for deletion, as well as ascheme value i=0, which targets a primary table entry.

As will be recalled, such a function can delete the primary table entry,automatically create secondary table aggregate data, and convert theprimary table into a “proxy” entry, or utilize an existing entry as aproxy for such aggregate data. Following such a function, theapplication can be finished (1306) (e.g., can be ready to operate inresponse to more events).

In this way, aggregate data can be rapidly generated as CAM look-ups areutilized, while at the same time needing far fewer CAM entries, throughthe use of “proxy” entries.

Another application event can include the arrival of a new packet(1308). In the very particular example of FIG. 13, in response to newpacket, an application 1300 can determine if the packet signifies theend (or reset) of a flow (1310).

If the new data packet indicates the end of a flow (Y branch from 1310),the application 1300 can delete the flow from the primary table. In theexample of FIG. 13, this can be accomplished by calling the GDeletefunction (1312) using a key value generated from the new data packet, aswell as a scheme value i=0 to designated the primary table. Theapplication can then be finished (1306).

If the newly arrived packet does not indicate the end of a flow (Nbranch from 1310), the application 1300 can look in the primary table tosee of the packet represents an existing flow, or represents a new flow.In the example of FIG. 13, this can be conveniently accomplished bycalling the GLookUp function (1314) using a key value K generated fromthe new data packet, as well as a scheme value i=0 to designated theprimary table. Such a function can use the data of the new packet(NEW_DATA) as new associated data (in the case of a new flow), or addsuch data to existing associated data (in the case of an existing flow).The application can then be finished (1306).

Another application event can include the deletion of a secondary tableentry (1316). As but one example, such an event can occur whenaggregated flow data is been exported for analysis.

In such an event, an application 1300 can execute the GDelete function(1318), utilizing a key value K to identify the entry for deletion, aswell as a scheme value i, which targets a secondary table.

As will be recalled, such a function can set a particular valid bit fora primary table entry that corresponds to the secondary table. Further,aggregated data for the secondary table can be read for export.Following such a function, the application can be finished (1306).

In this way, secondary table “entries” can be deleted by settingparticular bit values of a “proxy” entry.

The various functions described above can be arranged into a hierarchyas shown in FIG. 14. The hierarchy 1400 shows the various functionsdescribed above. In the particular arrangement shown, level 1 function1402 can be provided at a device level. As but one example, suchfunctions can be provided by hardware on a CAM device or section.

Level 2 functions 1404 can be provided at routine level. As notedpreviously, such functions can be provided by hardware, software or somecombination thereof. More particularly, one or more such functions (orsteps executed by the functions) may be provided by an ASIC or logicseparate from a CAM device, or a logic section incorporated in to a CAMdevice, or by steps executed by a processor according to an instructionset.

Level 3 functions 1406 can be provided at an application level. Forexample, a system may perform the functions in combination with otherhardware components. More particularly, a router or switch system mayprovide network flow monitoring and data aggregation with such anapplication.

Performance Examples

Having described both devices and methods according to the presentinvention. Performance examples of data aggregation according to theabove embodiments will now be described. The following examples arebased on assumed network traffic values. These values have beengenerated from an empirical study shown in “Their share: diversity anddisparity in IP traffic”, by Broido et al., and are as follows:

Average Number of Packets per Flow = 17 Average Bytes per Packet = 576Average Bytes per Flow = Average Number of Packets per Flow * AverageBytes per Packet = 9792These values can be used to calculate flow update rates for various linespeed values. A tabulation of such calculations is shown in FIG. 15. Tobetter understand how such values are generated, the calculations forone line of the table (line speed of 10 Gbps) is shown below:

Line Rate = 10 Gbps Average Packet Size = 576 bytes (from Broido et al.)= 4608 bits Average Packets per flow = 17 (from Broido et al.) AveragePackets per second = Line rate/bits per packet = 10 Gbps/4608 bits = 2.1Mpackets/s Average Flow Rate = Packets per second/Packets per flow = 2.1Mpackets/s/17 Packets per flow = 124 k flows/s Average Update Rate =Entry addition rate + Entry deletion rate = 124 k + 124 k = 248 kupdates/sReferring to FIG. 15, as line speed is increased, average update ratescan rise considerably. In the case of hashing, non-deterministic searchoperations particularly in the case of “funneling” and the like, may notbe capable of meeting the very fast look-up rates needed.

Referring now to FIG. 16, a table is shown illustrating the number offirst level (device) functions needed to execute the various secondlevel (routine) functions described above.

Using the information from table of FIG. 16, a needed rate for a fastestillustrated line speed can be calculated as follows. This exampleassumes that the number of aggregation schemes is eight (i.e., m=8), andthat the protocol is an Ethernet type protocol. Further, the rate atwhich secondary table entries are deleted is assumed to be an order ofmagnitude less than the rate at which primary table entries are deleted.This value is used as the system implements data aggregation, thusreducing the rate at which aggregate data is collected. It is assumedthat such a reduction is not only possible, but also a desirable featureto a network administrator.

Using such assumptions the following calculations can be made for a linerate of 10 Gbps running Ethernet:

Peak Packets per Second = 60 Mbps Primary Table look-up rate = PeakPackets per second = 60 Mpps Primary Table insert rate = Average FlowRate = 500 k (0.5 M) Primary Table deletion rate = Average Flow Rate =500 k (0.5 M) Secondary Table deletion rate = 50 k (0.05 M)The above calculations can be used to generate the table of FIG. 17.FIG. 17 shows the total rate at which device operations (i.e., level 1functions) need to be executed to meet the 40 Gbps line rate exampledescribed above. As shown, 66 million searches per second would beneeded to meet update speeds for such a fast line speed. While such arate might be difficult and/or complex to meet with a hashing solution,a high performance CAM device is capable of such high search rates. Asbut one example, the Ayama 10000 NSE manufactured by CypressSemiconductor Corporation of San Jose, Calif., is capable of performing266 million searches per second, easily meeting the needed high searchrate.

While a high performance CAM device can meet the above noted CAM accessrates, RAM access rates could present another bottleneck. However, anumber of different approaches can be used to increase overall RAMaccess rate. Two possible examples are shown in FIGS. 18A and 18B.

FIG. 18A is block diagram showing the same general components as that ofFIG. 4. However, unlike FIG. 4, a system 1800 can include multiple RAMinterfaces 1812, each corresponding to a different table (only four areshown in this example). In this way, a write/read to one table need notbe completed before the next table can be accessed. Such an arrangementcould meet any bottleneck presented by lower RAM access speeds.

FIG. 18B is block diagram showing the same general components as that ofFIG. 4. However, unlike FIG. 4, a system 1850 can include a RAM section1854′ composed of multiple banks (only two are shown in this example).In such a “bank select” arrangement, tables can be replicated intodifferent banks within the same RAM device. In this way, a write/read toone table in one bank, can be followed by a read/write to a next tablein a different bank. This can also lead to faster access rates.

Of course, one skilled in the art could arrive at various conventionalapproaches for improving RAM access speeds to meet the fast searchspeeds of a CAM section.

System and Device—Type of Service

A system for providing packet forwarding functions according to aservice type is set forth in a block diagram in FIG. 19 and designatedby the general reference character 1900. A system 1900 can include thesame general components as the system of FIG. 2. However, unlike thearrangement of FIG. 2, in FIG. 19 each physical entry of CAM portion1902 can store key searchable data for forwarding packet data.

In this case, such data can be a destination address of a packet, forexample. By use of global masking and service defining bit values, suchentries can serve as key data for multiple service types of a samedestination. As a result, in FIG. 19, a system 1900 can include onetable for destinations regardless of service type, instead of multipletables (one table for each service type).

In a preferred arrangement, a single bit (e.g., valid bit) candistinguish between each different service type.

An associated data portion 1904 can store associated data for allservice type tables. For example, associated data portion 1904 caninclude first forwarding data area 1908-0 for storing forwarding data(e.g., next hope information) for a first service type. Data areas1908-1 to 1908-m can store associated data corresponding to “m” otherservice types.

As is well understood, a different service type may provide a higher orlower quality of service, or be designated for predetermined data packetpayload (e.g., audio, video, etc.).

As in the case of FIG. 2, an associated data portion 1904 can preferablybe formed from RAM devices, even more preferably from DRAM, which mayprovide an advantageously low cost per bit.

In this way a system 1900 can provide high look-up rates (CAM speed) toassociated data in a tables totaling (m+1)*N RAM entries, with onlyabout N CAM entries.

To better understand service type lookups according to the presentinvention, one very particular example is shown in FIGS. 20A and 20B.

FIG. 20A shows a conventional approach for storing service values in aCAM section 2056 and associated data in a RAM section 2058. Thus, CAMsection 2056 includes search tables 2056-0 to 2056-3, each of whichstores lookup data for identifying a destination (in this case IPdestination address “D_IP”) and corresponding service type (TOS). In thevery particular example of FIG. 20A, search tables 2056-0 to 2056-3include four service types (1-4).

Forwarding associated data are also shown in FIG. 20A. In particular,associated data tables 2058-0 to 2058-3 are shown, which correspond tosearch tables 2058-0 and 2058-3. In the very particular example shown,each associated data table (2058-0 to 2058-3) stores a nexthop addressfor forwarding a packet according to the corresponding service type.

FIG. 20B shows how the data illustrated in the conventional case of FIG.20A can be more compactly fit into a single CAM table. Thus, FIG. 20Bshows only one table 2076-0 in CAM section 2076 containing the fourentries, each storing one of the destination addresses of FIG. 20A. Eachentry also defines one or more types of service for the destinationaddress, shown by valid bits “V0” to “V3”.

Thus, each physical CAM entry can serve as a “proxy” entry to representan additional service type. This is shown by a valid bit (V0 to V3)being set to “1” for these entries.

A RAM portion 2078 of FIG. 20B can be essentially the same as that ofFIG. 20A.

Of course, FIG. 20B represents but one possible application of thepresent invention.

Second Level Functions—Type of Service.

The above first level device functions and exemplary components can alsobe utilized by functions related to type of service applications, asdescribed below and with reference to FIGS. 21 and 22.

FIBLookUP

A “FIBLookup” function can be simpler than a “GLookup” function, likethat shown in FIG. 10, as aggregation is not included. A FIB Lookup candescribed according to the following syntax:

FIBLookUp(K, i, NEW_DATA)

Inputs: K=Search Key; i=type of service (TOS) (i^(th) global mask inCAM);

-   -   NEW_DATA=associated data of a packet.        In one arrangement, this function can include performing a first        level function NSearch to acquire output values “hit” and        “index”. A function can then determine if a matching entry has        been found from the search. This can include examining a “hit”        output value, and determining if it indicates a hit (hit=HIT).

If the search operation yielded a matching entry, then an entry existsin the CAM section that matches the new key K. If the search operationdoes not yield a matching entry, then no entry exists for the data, anda new entry must be created within CAM section with corresponding databeing written into a RAM section. In one arrangement, this can beaccomplished with another second level function FIBInsert(K, i,NEW_DATA), described in more detail below. Following such an insertoperation, the FIBLookUp function can end.

FIBDelete.

FIG. 21 shows one example of a CAM entry delete function “FIBDelete”. AFIBDelete function can be described according to the following syntax:

FIBDelete(K, i)

Inputs: K=Search Key; i=TOS (i^(th) global mask in CAM).

Referring to FIG. 21, a FIBDelete function is designated by the generalreference character 2100 and can include executing a search in a CAMsection with a key K and TOS value i (2102). As shown in the figure,this can include performing a first level function NSearch to acquireoutput values “hit” and “index”.

The function 2100 can then determine if a matching entry has been foundfrom the search (2104). This can include examining a “hit” output value,and determining if it indicates a hit (hit=HIT).

If the search operation does not yield a matching entry (N branch from2104), then no entry exists for the key value. This indicates an errorcondition (2106). As a result, an error indication can be returned to arequesting device.

If the search operation yields a matching entry (Y branch from 2104),then an entry exists in the CAM section that needs to be deleted, and aTOS value “i” can be checked to determine if the entry has any othervalid bits set (2156). If the to-be-deleted entry does not include anyother set valid bits (N branch from 2156), indicating the entry does notrepresent a pointer to another TOS, then entry can be “deleted”.

If the to-be-deleted entry includes another set valid bit, the CAM entryis representing another data set. Thus, “deletion” of the entry caninclude setting valid bit for the corresponding TOS indicator to anon-active value (in this case 0) (2108). In the example shown, such anoperation can include setting the appropriate valid bit to zero based onthe received value of “i”: Set Vld_bit[i]=0. A new key value can then bewritten into the CAM entry with such a changed valid bit value utilizingthe first level CAM write function: NWrite(index, Vld_bit[i]). Thisfunction can be a “masked” write, that only writes the “0” value to thevalid bit location corresponding to the “i” value. FIBInsert.

FIG. 22 shows one example of a CAM entry insert function “FIBInsert”. Itwill be recalled that such a function can be included in the FIBLookUpfunction described above. A FIBInsert function can be describedaccording to the following syntax:

Success=FIBInsert (K, i=0, NEW_DATA)

Inputs: K=Search Key; i=TOS;

-   -   NEW_DATA=associated data

Outputs: Success=a flag indicating success/failure of insertion of entry

Referring to FIG. 22, a FIBInsert function can first check for CAMentries that match a value to be inserted (2202).

The function 2200 can then determine if a matching entry has been foundfrom the search (2250). This can include examining a “hit” output value,and determining if it indicates a hit (hit=HIT).

If a search yields no match (N branch from 2250), then no entrycurrently exists that matches the newly arrived data. A CAM section canthen be examined for an available entry. This is shown in steps 2210 to2216. In one arrangement, these steps can be essentially the same assteps 1210 to 1216 of FIGS. 12A and 12B. Accordingly, a description willbe omitted. It is noted, however, that NWrite and DWrite functions writedata based on the i-th bit location, not a value “i=0”.

Referring still to FIG. 22, if a search yields a match (Y branch from2250), the matching entry can then be checked to see of the i-th bit hasalready been set. If the i-th bit has been set (Y branch from 2252), anentry already exists for K,i combination, thus indicating an error(2254). If the i-th bit has not been set (N branch from 2252), an entrycan be inserted at the location indicated by the returned index value.This is shown in steps 2236 to 2240. In one arrangement, these steps canbe essentially the same as steps 1236 to 1240 of FIG. 12B. It is noted,however, that NWrite and DWrite functions based on the i-th bitlocation, not a value “i=0”.

It is understood that while the various embodiments describe particulardata aggregation approaches, such approaches should not be construed aslimiting the invention to any particular protocol, collection of keyfields or numerical operation. Aggregation of data according to thepresent invention can generally be considered searching a data sourcefor a set of like entities to form a single entity, and arriving at somefigure, such as a sum, count or average. An aggregation can be derivedby defining rules, or keys, that can determine likeness of dataentities. Thus, the present invention should not be construed as beinglimited to summation as the only aggregation method.

Various embodiments shown above have illustrated a primary table formedin a CAM portion with “N” entries (corresponding to “m” different Nsized secondary tables for associated data). However, in operation, thenumber of entries within a CAM portion may need to exceed N. Such anevent can occur when an aggregation entry exists for which there is nocorresponding primary table entry. That is, a secondary table may bestoring “stale” aggregate data. The number and frequency at which avalue N may be exceeded can be dependent upon the particular network inwhich the invention is employed. However, such a number is expected inmost applications to be relatively small, as aggregated data values arenot expected to become stale often, as such operations work against thebasic motivation for data aggregation.

Accordingly, it may be desirable to provide a CAM portion having alimited number of extra entries for accommodating cases in which staleaggregate data arises.

Alternate Embodiment—No Valid Bits.

While the above embodiments have described data aggregation schemearrangements in which CAM section entries include valid bits for eachscheme, an alternate embodiment may not include such valid bits. Such analternate embodiment would utilize an offset address generator asdescribed above to access secondary table data based on a scheme value“i”. Such an approach could dispense with an NRead type operation in aGInsert function.

According to the above embodiments of the present invention, fast,deterministic data aggregation can be accomplished with considerablyless CAM memory than conventional approaches, like those describedabove. Utilizing less CAM resources can result in a less expensivesystem.

Still further, because the embodiments can provide deterministicresults, the disadvantages associated with collisions do not exist.Along these same lines, the approaches can be implemented without theneed for know how regarding hashing. There is no need to worry aboutselecting an appropriate hashing function to ensure low collisionoperations for a given network. It follows that by avoiding delaysintroduced by collisions, the present invention can provide higherthroughput rates than hashing approaches.

The above embodiments can provide a single search interface. Thus,second level functions can be conveniently called to perform essentiallyall the necessary tasks for executing data aggregation.

The above embodiments can provide a system with more flexibility. In theevent a CAM section has extra available entries, such entries can beused for other types of searches.

For this reason, the various embodiments of the present invention canenjoy wide application in various industries. As but one example, thesystems and methods according to the present invention can be includedin network routers or switches to provide data aggregation capabilitieswithout the throughput limitations of hashing approaches, or the highcost of implementing individual primary and secondary CAM tables.

It is also understood that the embodiments of the invention may bepracticed in the absence of an element and or step not specificallydisclosed. That is, an inventive feature of the invention can beelimination of an element.

Accordingly, while the various aspects of the particular embodiments setforth herein have been described in detail, the present invention couldbe subject to various changes, substitutions, and alterations withoutdeparting from the spirit and scope of the invention.

1. A method of providing multiple data sets for search on a single physical set of storage entries, comprising the steps of: storing at least one data value of a first data set in one physical entry of a plurality of globally maskable content addressable memory (CAM) entries; storing at least a second data value of a second data set, different form the first data set, in the one physical entry; and distinguishing between the first and second data value by applying a predetermined global mask to the one globally maskable CAM entry.
 2. The method of claim 1, further including: storing associated data for each data value of the first data set in a first memory section starting at a first offset location; and storing associated data for each data value of the second data set in a second memory section having different addresses than the first memory section, the second memory section starting at a second offset location.
 3. The method of claim 1, wherein: the first data set includes a network flow primary table and the second data set comprises at least one aggregation scheme; the step of storing at least one data value of a first data set in the one physical entry includes storing network flow information in corresponding physical CAM entries to form network flow entries of the primary table; the step of storing at least a second data value in the one physical entry comprises using an existing primary network flow entry as an aggregation entry that corresponds to the at least one aggregation scheme; and storing associated data for each flow entry in a first associated data area, storing associated data for each aggregation scheme in a different associated data area.
 4. The method of claim 3, wherein: storing network flow information further includes setting a primary table valid bit in the physical entry for each network flow entry.
 5. The method of claim 4, wherein: the primary table has primary table entries that each include “k” data fields, where k is an integer greater than 1; each aggregation scheme includes a subset “p” of the k data fields, where p<k; and distinguishing between the first and second data value includes searching while applying a scheme global mask value that masks all but the p data fields of the k data fields to search the aggregation scheme, and searching without the scheme global mask value to search the primary table.
 6. The method of claim 5, wherein: the global maskable CAM entries store a primary table and a plurality of aggregation schemes; and the step of storing network flow information includes receiving new network flow information, searching for an existing aggregation scheme entry having a subset of fields that match the new network flow information fields, and if writing the new network flow information would not alter a field value combination utilized by the existing aggregation scheme entry, storing the new network flow information in the matching CAM entry, and if writing the new network flow information would alter the field value combination utilized by the existing aggregation scheme entry, storing the new network flow information in an unused CAM entry, if an unused CAM entry is available.
 7. The method of claim 3, further including: receiving a delete command to delete an existing primary table entry and aggregate the data of the primary table entry; in response to the delete command for each of a plurality of aggregation schemes, searching for an existing aggregation entry having a subset of fields that match corresponding fields of the primary table entry, and if an existing aggregation entry cannot be found that has the subset of fields that match corresponding fields of the primary table entry, converting said network flow entry into an aggregation scheme entry.
 8. The method of claim 3, further including: further distinguishing flow entries from aggregation entries according to valid bits in each entry.
 9. The method of claim 8, wherein: the valid bits include at least one unique bit location corresponding to a primary table for each flow entries, and at least one bit location unique to each aggregation scheme.
 10. The method of claim 9, further including: accessing associated data for each aggregation entry by generating an offset address according to a scheme value, and adding the offset address to an index value corresponding to a matching aggregation entry.
 11. The method of claim 1, wherein: the first data set includes a first type service designation for a data packet and the second data set includes a second type service designation for the data packet; and the step of applying the predetermined global mask includes masking all but a desired service type designation to generate packet forwarding data corresponding to the desired service type.
 12. The method of claim 11, wherein: each CAM entry includes a predetermined number of service type bit locations, each bit location corresponding to a different service type designation, each service type bit location having a set state and not set state.
 13. The method of claim 12, wherein: the step of storing at least one data value includes setting a predetermined service type bit for a plurality of entries to a set state to establish a default service value for the data values of the entries.
 14. The method of claim 12, wherein: each CAM entry further includes at least a destination field that indicates the destination of a data packet.
 15. The method of claim 11, further including: receiving a delete command to delete an existing first data set entry, the delete command indicating matching field values and at least one service type; and in response to the delete command finding each first data set entry having fields that match the matching fields of the delete command, and if the service type designations of the entry do not match all of the at least one service type of the delete command, setting the matching service type designations to a non-set state, if all of the service type designations of the entry match the at least one service type of the delete command, deleting the entry.
 16. The method of claim 11, further including: accessing associated data for each aggregation entry by generating an offset address according to the service type designation, and adding the offset address to an index value corresponding to the service designation.
 17. The method of claim 11, further including: receiving a write command indicating matching field values and at least one service type; and in response to the write command finding an entry having fields that match the matching fields of the write command, and if the service type designations of the entry do not match the service type of the write command, setting the matching service type designation of the entry to a non-set state.
 18. A method for providing multiple data set values with a single set of globally maskable content addressable memory (CAM) entries, comprising the steps of: storing first data set values in the CAM entries, each data set value comprising a plurality of unique fields; and utilizing a CAM entry that stores a first data set value to simultaneously store a proxy value for at least one other data set, each proxy value representing a data set value that shares a subset of the unique fields of the first data set values.
 19. The method of claim 18, wherein: storing first data set values includes storing associated data for each first data set value in a first random access memory (RAM) section accessed with a first table offset address value; and storing associated data for other data set in a corresponding other RAM section accessed with other offset address values unique to each other data set.
 20. The method of claim 18, further including: in one type of read function, reading RAM data according to the first table address offset values and an index value generated by searching the first data set values; and in another type of read function, reading RAM data according to one of the other address offset values and an index value generated by searching proxy values for the other data set of the other address offset value.
 21. The method of claim 18, further including: in one type of data modify operation, reading old data from a RAM location accessed by one of the address offset values and an index value, and modifying the old data with data from a newly received data packet; and in another type of data modify operation, reading old data from a RAM location accessed by one of the other address offset values, and modifying the data with other data read from a RAM location accessed by the first table address offset value.
 22. The method of claim 18, wherein: the first data set values are a primary network flow identification table having a first data set value corresponding to each flow of a monitored data network; and the at least one other data set value includes at least one data aggregation scheme that aggregates selected values of monitored flows.
 23. The method of claim 18, wherein: the first data set values are packet forwarding values corresponding to one type of network packet transmission service; and the at least one other data set values are packet forwarding values corresponding to at least one other type of network packet transmission service.
 24. A system for searching multiple data set values with a single content addressable memory (CAM) table, comprising: a CAM section having N globally maskable CAM entries, each CAM entry operable to store N data set values that index to first set associated data, at least one second set associated data, or both first and second set associated data; and a control section comprising, an address generator that generates offset addresses corresponding to a predetermined global mask value, the offset addresses including a first data set offset address and a plurality of secondary set offset addresses each corresponding to a different set of associated data.
 25. The system of claim 24, further including: an associated data section comprising a first section having no more than N entries that store first set associated data, and at least a second section having no more than N entries for storing second set associated data.
 26. The system of claim 24, wherein: the control section is selected from the group consisting of: an application specific integrated circuit formed on a different integrated circuit than the CAM section, a programmable logic device formed on a different integrated circuit than the CAM section, and a processor executing instructions.
 27. The system of claim 24, wherein: the control section is formed in the same integrated circuit as the CAM section.
 28. The system of claim 24, wherein: each CAM entry includes a plurality of valid bits; and the CAM section includes a global mask register that generates global masks for searching the CAM entries, the global mask register including a global mask corresponding to each valid bit.
 29. The system of claim 24, wherein: each CAM entry comprises a plurality of valid bits; and the control section further includes an input parse section that adds valid bits to received search key data to generate an internal search key applied to the CAM entries.
 30. The system of claim 24, wherein: the control section further includes an address modifier that generates a new address based on an offset address from the address generator and an index value from the CAM section.
 31. The system of claim 24, wherein: the data set values that index to the first set associated data and both the first and second set associated data comprises a primary table, where each data set value defines a network flow; and the data set values that index to the second set of associated data and both the first and second set associated data comprises an aggregation table, where each data set value defines a network flow data aggregation scheme.
 32. The system of claim 24, wherein: the data set values that index to the first set associated data comprises forwarding information for a first type of packet transmission service; and the data set values that index to the second set of associated data comprises forwarding information for a second type of packet transmission service. 